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1  Introduction 

This  note  describes  the  implementation  of  a  video  camera  interface  that 
performs  a  simple  form  of  real-time  image  data  reduction  expected  to  be 
useful  for  real-time  tracking  of  3-D  objects  by  observation  of  their  silhou- 
ettes. Given  an  image  containing  one  or  more  objects  and  given  a  particular 
object,  the  interface  will  extract  boundary  data  for  just  this  object  .The 
system  is  fast  enough  to  operate  in  a  real-time  environment  that  contains 
many  objects,  in  which  one  wants  to  keep  track  of  a  particular  object,  all 
of  which  may  be  translating  and  rotating  with  respect  to  each  other  (see 
appendix  B). 

2  Functional  Description 

Suppose  one  was  attempting  to  find  a  specific  object  in  a  given  image. 
For  example  in  Figure  1  there  are  three  objects:  Oi,  O^,  Oj,  and  we  may 
suppose  that  one  was  interested  in  object  Oj  only.  How  can  one  to  dis- 
tinguish object  O2  from  the  others  easily  and  rapidly?  The  special  image 
processing  board  we  describe  implements  this  capability  in  a  simple  way. 
To  explain  its  structure  we  assume  that  the  image  to  be  analyzed  consists 
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Figure  1:  A  sample  image  with  three  objects 


of  N  lines  with  M  pixeb  per  line,  and  that  the  image  is  a  binary,  i.e.  con- 
sists of  black  objects  on  a  white  background  or  white  objects  on  a  black 
background.  Figure  2(a)  shows  an  example  of  a  scan  line  (i.e.  a  row  of 
pixels)  in  which  two  black  segments  {Li,Ri)  and  (Lj.iJi)  appear.  Two 
points,  which  are  the  left  boundary  and  the  right  boundary  of  a  segment 
respectively,  are  enough  to  represent  a  segment.  In  an  image  with  a  white 
(resp.  black)  background  with  black  (resp.  white)  objects,  we  define  the 
'left  boundary'  of  a  segment  to  be  the  leftmost  position  in  the  segment  X 
at  which  a  transition  of  pixel  values  from  white  to  black  (resp.  black  to 
white)  occurs,  and  similarly  we  can  define  the  right  boundary  as  the  first 
position  after  the  left  boundary  at  which  a  transition  from  black  to  white 
(resp.  white  to  black)  occurs.  To  pick  out  a  specific  segment,  e.g.  (Li,  Ri), 
one  must  specify  a  control  point  (i.e.  reference  point)  X  on  its  scan  line. 
The  point  X  then  identifies  the  rightmost  whose  left-hand  boundary  lies  left 
(rather  than  right)  of  X.  For  example,  in  Figure  2(a)  the  point  X  marked 
identifies  the  segment  {Li,Ri)  while  in  Figure  2(b)  the  point  X  identifies 
the  segment ( Z/j,  iZ^)  .  In  Figure  2(c)  X  identifies  nothing  since  no  segment 
begins  left  of  the  control  point  X.  Thus  specifying  a  control  point  for  each 
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Figure  2:  Examples  of  Control  positions 

scan  line  appropriately  allows  us  to  pick  out  a  specific  object  from  a  group 
of  others  and  return  the  boundary  of  the  object. 

Our  hardware  realization  of  the  function  just  described  consists  of  three 
major  parts:  (1)  a  camera,  (2)  an  interface  board,  and  (3)  a  Motorola  68000 
microprocessor  (used  as  a  top  level  controller).  Figure  3  gives  a  system 
diagram.  Currently,  we  use  a  General  Electric  TN-2200  CID*  which  has  a 
frame  time  of  ^  of  a  second  and  generates  128  x  128  images  consisting  of 
8-bit  gray-scale  pixels.  The  task  of  the  interface  board  is  to: 

•  convert  the  grey-scale  image  to  a  binary  image 

•  find  the  left  and  right  boundaries  of  the  object  specified  by  the  con- 
trol point  (received  from  the  controlling  microprocessor)  in  each  scan 
line. (These  boundaries  are  a  pair  of  integers,  i.e  pixel  numbers,  held 
in  a  pair  of  registers  on  the  interface  board.) 

•  notify  an  attached  microprocessor  when  these  two  values  become 
available  for  a  scan  line,  and  transmitting  these  two  values  to  the 
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Figure  3:  System  schematic 


microprocessor. 
The  control  program  running  on  the  microprocessor  is  responsible  for: 

•  setting  the  threshold  value  for  binary  image  conversion  to  be  applied 
to  the  next  frame  generated  by  the  camera. 

•  supplying  sequence  of  'control  points'  one  for  each  8caa.line  to  be 
used  in  the  manner  explained  above,  (actually,  the  next  scan  line), 

•  saving  the  left  and  right  segment  boundaries  generated  for  each  scan 
line  by  the  interface  board. 

Figure  4  represents  the  interface  board  schematically.  The  binary  image 
generator  receives  the  stream  of  video  signals  generated  by  the  camera  .  An 
Analog  to  Digital  converter  then  converts  the  incoming  signal  of  from  analog 
to  a  six  bit  binary  number.  A  clock  pulse  synchronizes  the  A/D  conversion 
with  the  rest  of  the  interface  board's  digital  activity.  Each  digitized  pixel 
generated  by  the  A/D  converter  is  compared  with  a  threshold  value  stored 
in  TRESIIOLD  register,  producing  binary  output  stream.  A  binary  counter 


synchronized  with  this  binary  pixel  generation  process  starts  counting  at 
the  beginning  of  each  line.  A  second  comparator  compares  the  control 
point  X  (suplied  by  the  attached  microprocessor)  to  the  current  value  of 
this  counter  and  sends  a  boolean  signal  XLTC  (  X  less  then  counter  )  to 
the  logic  unit.  The  logic  unit  generates  one  of  two  binary  signals  LOADJL. 
or  LOAD-R  in  a  manner  determined  by  a  set  of  flags  reflecting  the  current 
logical  context.  These  flags  have  the  names:  pixel  value,  last  pixel  value, 
reverse  mode,  LJs_8et,  RJs-set,  and  XLTC,  from  which  the  reader  will  be 
able  to  construe  their  logical  significance.  When  the  signal  LOADX  (reap. 
LOAD_R)  is  generated,  the  current  value  of  the  pixel  counter  is  loaded 
into  the  L  (resp.  R)  register.  At  the  end  of  each  line  the  values  in  L  and 
R  are  moved  into  a  pair  of  buffer  registers  called  L.buffer  and  R.buffer 
.  Another  part  is  the  interrupt  logic  which  the  interface  board  uses  to 
alert  the  attached  microprocessor  when  values  in  L.buffer  and  R_buffer 
are  ready  for  transmission.  At  the  beginning  of  each  framej_the  signal 
VD  (vertical  drive)  from  camera  sets  bit  0  of  the  interrupt  status  register, 
and  at  the  end  of  each  line,  the  signal  LD  (line  drive)  from  camera  sets 
bit  1  of  the  interrupt  status  register.  When  either  bit  of  interrupt  status 
register  is  set,  an  interrupt  request  signal  will  be  generated  for  the  attached 
microprocessor.  All  the  registers  are  connected  to  the  microprocessor  in  a 
memory  mapped  I/O  fashion.  The  interrupt  register  is  a  read-only  register, 
in  the  sense  that  any  write  operation  to  this  register  will  clear  all  bits  of  the 
register  and  the  interrupt  request  signal  will  also  be  cleared.  More  details 
about  the  logic  unit  of  the  interface  board  can  be  found  in  Appendix  B. 
Additional  details  concerning  the  interface  board  design  can  be  deduced 
from  the  following  detailed,  low-level  simulation  of  it's  logical  action. 
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Algorltlim: 


/  IR  —       interrupt  status  register        / 

/  bit  0  >  1,  indicate  the  new  frame  / 

/  bit  1  -  1,  indicate  the  new  line  / 

/  THRESHOLD  —  register  to  hold  the  threshold  value  / 

/  for  converting  a  general  image  to  a  / 

/  binary  image  ;  / 

/  REVERSE  —   flag  indicating  the  mode  of  image     / 

/  0,  white  background  ;  / 

/  1.  black  background  ;  / 

/  X  —  register  to  hold  the  control  point  :        / 

/  X' —  buffer  to  hold  the  control  point  for  next  line  / 

/  L  —  register  to  hold  the  current  left  boundary  found  / 

/  R  —  register  to  hold  the  current  right  boundary  found/ 

/  L'~  buffer  to  hold  the  left  boundary  of  last  line  / 

/  R'—  buffer  to  hold  the  right  boundary  of  last  line  / 

/  PIXEL  —      register  to  hold  the  value  of  current  pixel/ 

/  OLD.PIXEL  —  register  to  hold  the  value  of  last  pixel  / 

/  L_is_set  —   flag  indicating  that  L  has  been  set  / 

/  R_i8_set  —   flag  Indicating  that  R  has  been  set  / 

/  TIR  —       temporary  interrupt  register/ 


processl  :  /executed  by  the  interface  board/ 

input:   TRESHOLD . REVERSE :    /from  microprocessor  for  every 

frame/ 
X;  /from  microprocessor  for  every 

line/ 
output:  L'.R*  /to  microprocessor  for  every 


line/ 


begin 

if  (new 

.frame)  then 

begin 

IR(0) 

:-  1  : 

X'  :- 

R  :-  L  :-  0: 

end 

if  (end 

_of_a_line)  then 

begin 

IR(1) 

:-  1  : 

L'  :  = 

L  : 

R"  :- 

R  : 

L  :- 

R  :-  0  : 

X  :- 

X*  : 

OLD  PIXF.T.  :=•  REVERSE 

L_ia_ 

set  :=■  0  : 

R_i8_ 

set  :=>  0  : 

/set  frame  interrupt  to  signal 
microprocessor/ 
/initialize  registers  / 


/set  line  interrupt  to  signal 

microprocessor/ 
/store  the  left  and  right  boundary 

of  segment  into  buffers/ 
/initialize  L  and  R  registers  / 
/load  the  control  point  register 

from  buffer/ 
/set  OLD.PIXEL  to  background,  for 

the  first  pixel  in  the  next  line/ 

/initialize  two  flags  / 
for  piiel.count  :»  1  to  M  do  /loop  for  each  pixel  in  line/ 
begin 

OLD.PIXEL  :=■  PIXEL  :        /backup  last  pixel/ 
set  PIXEL  with  digitized  new  pixel  ; 
if  (pixel_count  <=  X)  and 

(((REVERSE  -  0)  and  (OLD.PIXEL  -  0)  and  (PIXEL  -  1)) 
or 

((REVERSE  -  1)  and  (OLD.PIXEL  -  1)  and  (PIXEL  -  0))) 
then      /  LEFT  BOUNDARY  / 
begin 

L  :-  pixel.count  :         /load  left  boundary  register/ 
L.is.set  :-  1  :  /set  the  flag/ 

end 


else 

if  (L.ls.aet  -  1)  and  (R.is.set  -  0)  and 

(((REVERSE  -  0)  and  (OLD.PIXEL  =  1)  and  (PIXEL  =  0)) 
or 

((REVERSE  -  1)  and  (OLD.PIXEL  -  0)  and  (PIXEL  -  1))) 
then       /  RIGHT  BOUNDARY  / 
begin 

R  :■  pixel.count  ;  /load  right  boundary  register/ 

R-is.set  :«  1  ;  /set  the  flag/ 

end  : 
end  ;  /for  loop/ 
end  ;  /processi/ 


process2  :   /executed  bj  the  microprocessor/ 


input:   L'.R': 

output:   TRESHOLD . REVERS : 

X': 

begin 

waiting  for  IR  to  be  set  ; 

TIR  :-  IR  : 

IR  :-  0  : 

if  (TIR(O)  -  1)  then 


/from  interface  board  for  every 

line/ 
/to  interface  board  for  every 

frame/ 
/to  interface  board  for  every 

line/ 

/waiting  for  interrupt/ 
/read  interrupt  status  register/ 
/clear  interrupt  source/ 
/frame  interrupt/ 


9 


begin 

set  THRESHOLD  with  Initial  value  ; 

set  REVERSE  to  proper  mode  : 

set  X'  with  Initial  value  : 
end 
else 

if  (TIR(l)  -  1)  then  /line  interrupt/ 

begin 

read  L'  and  save  in  memory: 

read  R'  and  save  in  memory: 

calculate  a  new  control  point  and  store  into  X': 
end: 
end  ;  /proce882/ 
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APPENDIX  A 


APPLICATION  OF  THE  INTERFACE  BOARD 


The  next  paragraph  sketches  an  algorithm  which  tracks  one  moving  object 
in  a  scene  containing  many  moving  objects.  Assuming  that  we  have  the 
object's  boundary  from  the  previous  frame,  the  key  question  is  how  to  use 
this  information  to  find  the  boundary  of  the  same  object  in  the  next  frame. 
To  do  this  we  note  that  the  motion  of  the  object  is  continuous,  and  so 
that  since  we  are  working  in  real-time,  the  object  can  move  at  most  some 
limited  number  K  of  pixels  (in  any  direction)  between  two  successive  frames. 
Exploiting  this  fact  in  the  simplest  way,  we  build  the  smallest  horizontal 
rectangle  around  the  object  such  that  the  distance  from  the  rectangle  to 
any  point  in  the  object  is  at  least  K  pixels.  Our  object  must  be  found 
within  this  rectangle  in  the  next  frame,  and  so  we  set  the  (hardware  board) 
control  point  equal  to  the  right  edge  of  the  rectangle,  ignoring  any  object 
positioned  to  the  right  of  the  rectangle;  also  we  ignore  any  object  positioned 
either  above  or  below  the  rectangle.  It  then  only  remains  to  restrict  all  L 
and  R  values  read  from  the  board  to  those  that  lie  to  the  right  of  the 
rectangle's  left  edge.  Since  the  maximum  motion  of  the  object  is  K  pixels 
per  frame,  the  algorithm  assumes  that  that  no  other  object  is  ever  within 
K  pixels  from  the  monitored  object. 
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Figure  5:  Top  view  of  the  board 
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Figure  7:     SchejtHtic  diagram 
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